Systematicity and Idiosyncrasy in Iconic Gesture Use: Empirical Analysis and Computational Modeling
نویسندگان
چکیده
Why an iconic gesture takes its particular form is a largely open question, given the variations one finds across both situations and speakers. We present results of an empirical study that analyzes correlations between contextual factors (referent features, discourse) and gesture features, and tests whether they are systematic (shared among speakers) or idiosyncratic (inter-individually different). Based on this, a computational model of gesture formation is presented that combines data-based, probabilistic and model-based decision making.
منابع مشابه
Systematicity and Idiosyncrasy in Iconic Gestures—A Probabilistic Model of Gesture Use
Introduction Current literature on gesture research states that the question “why different gestures take the particular physical form they do is one of the most important yet largely unaddressed questions in gesture research” [1, p. 499]. This holds especially for iconic gestures, for which information has to be mapped from some mental image into (at least partly) resembling gestural form. Thi...
متن کاملHow Do Iconic Gestures Convey Visuo-Spatial Information? Bringing Together Empirical, Theoretical, and Simulation Studies
We investigate the question of how co-speech iconic gestures are used to convey visuo-spatial information in an interdisciplinary way, starting with a corpus-based empirical and theoretical perspective on how a typology of gesture form and a partial ontology of gesture meaning are related. Results provide the basis for a computational modeling approach that allows us to simulate the production ...
متن کاملInterpretation of Shape-Related Iconic Gestures in Virtual Environments
So far, approaches towards gesture recognition focused mainly on deictic and emblematic gestures. Iconics, viewed as iconic signs in the sense of Peirce, are different from deictics and emblems, for their relation to the referent is based on similarity. In the work reported here, the breakdown of the complex notion of similarity provides the key idea towards a computational model of gesture sem...
متن کاملGrounding the Simulation of Iconic Gestures in Gesture Typology
How are co-speech iconic gestures used to convey visuo-spatial information? We investigate this question, which is still relatively unexplored [1], with an interdisciplinary methodology combining the empirical study of speech and gesture use, the elaboration of theoretical reconstructions and the formulation of generation models that enable the simulation of such communicative behaviour with vi...
متن کاملGNetIc - Using Bayesian Decision Networks for Iconic Gesture Generation
Expressing spatial information with iconic gestures is abundant in human communication and requires to transform a referent representation into resembling gestural form. This task is challenging as the mapping is determined by the visuo-spatial features of the referent, the overall discourse context as well as concomitant speech, and its outcome varies considerably across different speakers. We...
متن کامل